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ABSTRACT 

A computer system, IconiCase, is currently being 
developed to assist users in the task of classifying and memorizing 
medical cases. The system uses assemblies of icons, called Concept 
Graphics (CG) , as pointers to medical cases, leading to a form of 
electronic document delivery in which information is accessed at 
various levels, with a gradual transition between levels. Icons are 
designed to be interpreted without the help of textual explanations; 
however, when a new icon is introduced, its textual interpretation is 
automatically provided and its correspondence to the text of the case 
is highlighted. In the icon editor, new icons can be made by screen 
captures of basic shapes created with a standard painting or drawing 
program. It is argued that even short gaps in the way information is 
presented to users lead to cognitive difficulties that could be 
prevented by better human/machine interfaces; CG, as mnemonic 
devices, provide meaning that is remembered by the viewer on the 
basis of cues offered by the icons. In addition, the active role 
given to users in organizing their access to the information leads to 
more effective information retrievals Problems of using iconic 
compositions include: difficulty in interpretation, unclear meaning 
for a juxtaposition of several icons, i cons * . exces s ive use of 
valuable screen space, and cumbersome manipulation. (Contains 21 
references . ) (AEF) 
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Abstract. A computer system is proposed to assist humans in the task of classifying and 
rnemonzing medical cases with the help of assemblies of icons. caUed Concept GraphTcs 
(CG). This leads to a forni of Electronic Document Delivery in which infonnaS is 
accessed at vanous levels of details and the transitions between the levels is Tlso 
•MS argued that rapid review of large sets of documents for content and dff ereS ana^sl? 
.s a valuable activity that can be effectively supported by electronic systems ^ 
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Medi«^ arXh^MeJ^S 

rnemorizinglarg'en^mbL^sotmSSicCSgiSc^^^^^^^^^^^^ "^^^"^ categorizations and 
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difficulties associated with the use of icons. In the first part of this paper we define Concept Graphics and 
propose them as useful intermediaries between text and human mental representations. 

Concept Graphics 

In the following the detailed descriptions of medical pathologies that form a corpus of documents are caUed 
cases. Cases are obtained in the form of texts that can be very detailed, such as entries in The Merck Manual. A 
case has a unique textual identifier plus textual and iconic indexing terms. In addition it is represented by a 
composition of elementary icons, called a Concept Graphics (CG). that captures the main characteristics of the 
case, such as symptoms, treatments, factors such as age. habits, etc. Thus the CG components need not be 
homogeneous. The choice of these characteristics may reflect the biases of a particular expert, or general user of 
the system. An example of a CG corresponding to the case "acute pancreatitis" is given in Figure 1 . Note that 
the use of color make the CGs more easily identifiable.. 





Acute Pancreatitis 


The site of the major problem is the abdomen with severe 
pain, located centrally and radiating through to the back 

(3D image of the abdomen with arrow showing the 
direction of the pain). Manifestations of nausea and 
vomiting and accelerated pulse rate. The white blood 
cell (leucocyte) count is elevated and the concentration 
of of calcium ions is diminished. Acute condition based on 
genetic abnormalities. Predisposing condition for acute 

pancreatitis are akoholinsm and bilestones. 
Warning: condition may lead to shock. 







Figure 1. A concept graphic and associated short text 



For us CG are mnemonic devices, akin to the chimerical compositions used by medieval orators to 
summarize the main points of the sermon they were going to make. For a CG to be a mnemonic device means 
that its meaning is not just in the icons themselves, but that it is remembered by the viewer on the basis of 
cues offered by the icons. It should be noted that the early forms of Mesopotamia writing were assemblies of 
pictograms representing concrete things or actions (Bottero. 1993). It is only later that syllabic meaning was 
ascribed to pictograms which, by becoming more abstract, led to the alphabet. 

As much as possible icons are designed to be interpreted without the help of textual explanations; e.g. a 
broken bone, a brain, etc. However there is a limit to the complexity of meaning that can be expressed by- 
icons in the form of familiar concrete objects and abstract icons are a necessity. Our approach to resolving the 
problem of understanding unfamiliar icons is to link the individual icons as well are icon sequences to text and 
to introduces the use of new icons in a progressive manner to users. This means that what users are shown in a 
CG depends on what icons they understand. This also implies that a user model (Sleeman. 1984) be maintained 
bv the system. When a new icon is introduced its textual interpretation is automatically provided and its 
correspondence to the text of the case is highlighted. In Figure 1. the bolded letter words in the left text field 
correspond to the icons in the CG on the right. Later we provide additional details on how practically users arc 
made familiar with icons and iconic compositions, as well as the kind of assistance they can have when 
"reading" iconic compositions. 

Discounting for the time being the familiarity/unfamiliarity aspect, the advantage of using CG to access 
information is the exploitation the human visual processing abilities adapted to deal with objects in multiple 
dimensions such as space, shape, color, texture . There are several apparent consequences: 

- For the same speed of access, more information is obtained by the user than if it had been conveyed in textual 
form; or conversely the same amount of information is obtained more rapidly. Speed of access is an important 
factor when dealing with Uu-ge amounts of documents and traditional access methods through Boolean queries or 
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JlTatns Trtfv '"'f u °' cumbersome. Users may simply nol have the time available 
••n Jno ?K f ^"'^8 back to mind the content of a large set of documents a sort of rapid browsing called 

flying through hypertext" (Lai, Manber. 1991) is needed. ^ 

in w.'^rnf'l^eTf °^ 'co"ic,co"iPositions and the ability to scan through many cases in a short time result 
SrJ^rioo^fv!! ^'"^ 'l'"' ^^"^ 'f'^o^gh statistical classification methods 

aoDlSp .Jp IS"'' ''^^" "'""^ '"^'v^^' "^"'^ ^f'^' '0 Joo"^ either ahead of 

applying the method or in the inteipretauon of the results. Machine Learning algorithms (Kodratoff. 1988) and 
other discovery methods in databases, also fall under that criUcism. v^uuraiun. lyoo; ana 

- It may be the case that the composition of a CG conveys more meaning than just the sum of its parts. 

flwa^m' !Q'5n''°p!5 "''"^, compositions have been extensively studied and reported in the literature 
f- ^'^P?""?e"'s comparing the use of abstract icons and teu menus have concluded with no 
advantage for icons (Benbasat. Todd. 1993). Thus. v-unv-iuutu wim nu 

- interpreting individual icons can be difficult 

- it is not clear what meaning to ascribe to a juxtaposition of several icons 

- icons use a lot of valuable screen space. 

- icons and iconic compositions are cumbersome to manipulate. 



I Shape/Icon 

Capture from text Library g**^^ 
|CGEdilor|^ CGReader | ^ J 



pain 




abdomen pain in abdomen 



[CGOrganizerl 

Figure 2. Main modules of Iconicase Figure 3. Iconic composition 

The system IconiCasc is an attempt to capitalize on the advantages of using Concent Granhir<! while 

e3con?e;?r''h "^'^"^ '/"^"'^ the system IcSfc^e t'o emf^^^ 

tSo tn '""'^'"^ and analyzing large sets of cases (i.e. documents). The modules are 
SSion mode ^ corresponding to authoring/editing of CGs and of exploiting CGs in a 

Creating Concept Graphics 

Capture from Text 

Source text, such as entries in The Merck Manual, must be summarized in order to regroup the main features 
of a case and provide a textual explanation that closely match the corresponding CG When co3^^ 
module will offer a more efficient way of creating the summary text from the original texuE by TSd cm 
and paste operations; a user selection of words, group of words and sentences will £ seTto the s.m^l^ 
Zfs^o^^llti^^^^^ '^^'^^ autoSiSly'^c^atVLt be"S 

The icon editor 

Icons are grouped set of basic shapes that form a sort of collage. This means that basi- ^hanp^ are nm 
'ZT'^ "° be rectangles and that they include regions that are transparent. Sgie 3 Iws Uie E ThaS^^^ 
abdomen and "radiating pain" and the composite icon "pain radiatSg from center of aMomen" cons^P 
S^^jr"--- ^^^-•^^ ^^a^^- sha^^ri^c^^aT^L^^^^^^ 

Dro^m 'SL?e" ^ Tf'^^J'l '^P'"'"' °^ ^^^'^ ^bapes created with a standard painting or drawine 
program. TTiere is a tool for defimng particular regions of a captured image as transpareni or opaque with a color 
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the user can choose. Regrouping basic shapes, naming and indexing an icon is done in a standard way. It is also 
possible to create new icons by modifying existing ones retrieved through an icon browser or from existing 
Concept Graphics. 

The Concept Graphic editor 

Concept Graphics are assembled manually but we are planning to introduce some automated assistance in 
selecting the icons and assembling the CGs. For instance the keywords recognized form the text of a case can 
be automatically translated into icons appearing in the right field of Figure 1. Use of a hierarchical thesaurus 
(C6t6 32) is also considered, to select icons higher in the hierarchy when a recognizable medical term in the text 
does not have yet a conesponding icon. 

The relative placement of icons in a Concept Graphic can be (at least partially) automated by default or 
production rules. Default placement rules could be statistical. For instance place the new icon at the average 
position of that icon in already created CGs. Production rules would supersede that default placement to locate 
semanticaliy related icons at the same relative place in a CG. 

Though icons appear with irregular contours, they are included in defining rectangles. This makes it possible 
to apply a tool we have developed (Kaltenbach, 1991) to compact CGs in as little screen space as possible, 
while seeking to preserve the relative positions of the composing icons and avoiding the overlap of icons. 

It is also in the CQ editor that the correspondence between text and icons is established. It has the form of 
links that enable the simultaneous highlighting (by flashing) of corresponding parts of text and icon groups 
when a selection in the left text or right image field of Figure 1 is made by the user. A default selection of text 
or graphic objects is obtained by simply positioning the mouse cursor over the object. Other selection modes 
for more detailed or coarser selections arc also available. 

Once a CG is edited it is placed in a CG browser under an appropriate category. 




Figure 4. Viewing cases sharing attribute **abdomcn** with the case **acute pancreaiilis" 
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Accessing concept Graphics 



In addition to helping users understand a CG. IconiCase provides alternate ways of efficiently scanning 
through large collections of Concept Graphics. Current implementation includes a CG player, a textual find 
utility, a browser by semantic categories (e.g. heart diseases, pulmonary diseases...). 

the CG reader is a facility that automatically highlights the keywords, sentences in the left field of Figure 
1. together with corresponding icons in the right field. The speed of scanning can be controlled by the reader 

the CG finder is a facility that enables the user to select particular icons or icon components on the right 
01 Hgure 1. and obtain the text list or collection of CGs that share the selected features. Figure 4 gives the 
result of a search when the icon shape "abdomen" is selected. A zoom facility for the CGs in Figure 4 is 
projected. ° 

the CG player enables users to view CGs (Figure 1) in succession at a speed they can control. The 
sequences of CGs that can be viewed are defined using the CG sequencer. 

the CG sequencer currently permits the definition of CG sequences that correspond to- 

♦ user defined sequences by associating markers on CG seen as in Figure 1 or on clicking on CG as seen in 
Figure 4. ° 

♦ sequences corresponding to a semantic category in the CG browser 

♦ finally it is projected to cluster cases on the basis of more semantically related definitions of distance 
between cases. This may rely only in part on a measure of distance between individual icons. 

Any defined sequence can be reordered according to a measure generality for CGs. A detailed definition of this 
measure would make this paper too long, so we provide only the underlying idea. A partition of the set of icons 
(and icon components) available in the system is created under the equivalence relation that all the elements of a 
subset are related if they all index a set of cases that is maximal in the sense that no super set of cases is indexed 
by all these elements. The number of cases associated with an element of the partition of icons gives a measure 
comr^nmTcon? ^^^^ ge""ality of a case is obtained as the average generality of its 

The sets of cases as just defined can be ordered by inclusion and thus be accessed as the nodes of a graph A 
particular node is seen in our system as a sequence of CGs (Fig. 1) or as a two dimensional display (Fig. 4) 
The user can traverse hnks in that graph to refine or expand the generality of the component cases entering in a 
sequence, whde focusing on a set of icons of current interest In fact there are many ways in which significant 
navigation can be achieved in that graph and this wUl be the object of subsequent paper. 

Using IconiCase 

First we return to the question of how users get to understand individual icons and iconic compositions At 
J ^ may exploit an already processed collection of cases. By this we mean that thVcases have 

already been summarized and associated with CGs by someone else, acting as author. TTie user may then choose 

either to get explanations of complete CGs or to be shown no more than i new icons (i=l 2 ) per CG By 
'"r" °' fy selecting a component the user gets the correspondence "with the text (left 

field m Fig. 1) and a more fonnal definition of the icon m the text field below the CG in Fig.l. Explanations 

are also available for groups of icons when they are selected by the user 

orJ^! nJ^vnf J '° "'"'^ ^"^^^"^ °f CGs. The interpretation of the icons and 

™r^eV??.l ,o^w Pos^s/oblems; user made associations are in the essence of mnemonic 
approaches (Yate. 1966). In counterpart the practical value of the system IconiCase then hinges critically on 
how easy it is to create CGs. We have described some early steps in that direction. 

to al",iS'fhK r -P'^r'^^ ^ ' '° ""y- '"'^^^ co"''^ be used by students in order 
to assist them m reviewing for exams. 

There is another objective to the system IconiCase Note that when viewing a large number of CG it may 
no be necessary to understand the individual icons in order to make some inferences. For instance some overall 
Sowl d/e ttZ "^ 't^h'^'"' ' .large collection of CGs leading to the identification of causal rdationsh^ 
i iS^ ^ > The scanning of CGs could also be viewed as a complement to automated knowledge 
discovery approaches in databases. The rapid human scanning of cases obtained by the automated approaches 
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would further validate or invalidate the newly extracted rule. This is important because usually many rules are 
discovered by the automated discovery approaches and screening through them to get the really meaningful is a 
veiy tedious task. 

Future work and conclusion 

As an interface for the delivery of large corpora of documents, a system like IconiCase requires many 
ad^qptive features and much fine tuning to gain acceptance with a wide variety of users. We have explored some 
of these dimensions but much remains to be done. We plan to perform experiments with users to study the time 
needed to learn the meaning of icons and of groups of icons. Also we shall study possible tradeoffs between the 
depth of details and speed the CGs can be displayed to humans. The criterion of evaluation should how correctly 
cases are reminisced. 

The approach is of course not strictly limited to medical cases. Also it could be considered as an extension 
of and a complement to case-based reasoning systems (Hammond, 89). 
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